Skip to content

feat(migration): add webhook secretKey to authType migration in v1130#27438

Merged
yan-3005 merged 2 commits intomainfrom
ram/webhook-migrations
Apr 16, 2026
Merged

feat(migration): add webhook secretKey to authType migration in v1130#27438
yan-3005 merged 2 commits intomainfrom
ram/webhook-migrations

Conversation

@yan-3005
Copy link
Copy Markdown
Contributor

@yan-3005 yan-3005 commented Apr 16, 2026

Summary

On the 1.12.5 branch, migrateWebhookSecretKeyToAuthType was incorrectly cherry-picked from the certification refactor, causing Jackson deserialization failures since webhook.json on 1.12.x had no authType field. That migration was removed from v1125 and a reverse migration was added in v1126 to fix users who had already upgraded.

This PR ensures users who were on 1.12.4 or older and upgrade directly to 1.13 (skipping 1.12.5 entirely) still get the webhook migration. It adds migrateWebhookSecretKeyToAuthType to the v1130 migration, which converts:

"config": { "secretKey": "value" }

"config": { "authType": { "type": "bearer", "secretKey": "value" } }

This is the correct canonical format on main/1.13 — SubscriptionUtil, WebhookRecipient, and Fernet all read authType from the webhook config.

The migration is idempotent: rows that already have authType (no secretKey) are skipped.

Changes

  • v1130/MigrationUtil.java — added migrateWebhookSecretKeyToAuthType(Handle handle)
  • mysql/v1130/Migration.java + postgres/v1130/Migration.java — call the new method in runDataMigration()
  • v1130/MigrationUtilTest.java — 6 unit tests covering: no rows, no secretKey, empty secretKey, non-webhook destination, MySQL path (JSON + SQL verified), Postgres path (JSON + SQL verified)

Type of change

  • Bug fix

Checklist

  • I have read the CONTRIBUTING document.
  • I have added a test that covers the exact scenario we are fixing.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

@yan-3005 yan-3005 self-assigned this Apr 16, 2026
@yan-3005 yan-3005 added the safe to test Add this label to run secure Github workflows on PRs label Apr 16, 2026
Copilot AI review requested due to automatic review settings April 16, 2026 14:32
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a v1130 data migration to move legacy webhook secretKey configuration into the newer authType structure for event subscription webhook destinations, and introduces unit tests to validate the migration behavior across MySQL/Postgres.

Changes:

  • Add migrateWebhookSecretKeyToAuthType(Handle) to v1130 MigrationUtil to rewrite webhook destination config from secretKeyauthType: { type: bearer, secretKey }.
  • Invoke the new migration from both MySQL and Postgres v1130 migration runners with error logging.
  • Add a v1130 migration unit test suite covering no-op and migration paths for MySQL/Postgres SQL updates.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v1130/MigrationUtil.java Implements the webhook secretKeyauthType JSON rewrite + DB update logic.
openmetadata-service/src/main/java/org/openmetadata/service/migration/postgres/v1130/Migration.java Wires the new migration step into the Postgres v1130 data migration flow.
openmetadata-service/src/main/java/org/openmetadata/service/migration/mysql/v1130/Migration.java Wires the new migration step into the MySQL v1130 data migration flow.
openmetadata-service/src/test/java/org/openmetadata/service/migration/utils/v1130/MigrationUtilTest.java Adds tests validating the migration behavior and SQL selection for MySQL vs Postgres.

@gitar-bot
Copy link
Copy Markdown

gitar-bot Bot commented Apr 16, 2026

Code Review ✅ Approved 1 resolved / 1 findings

Adds webhook secretKey support to the v1130 authType migration and includes a new idempotency test, addressing the previously missing coverage for already-migrated rows.

✅ 1 resolved
Edge Case: Missing idempotency test for already-migrated rows

📄 openmetadata-service/src/test/java/org/openmetadata/service/migration/utils/v1130/MigrationUtilTest.java:28-42
The PR summary states the migration is idempotent (rows with authType already set are skipped), but there's no test covering a row that already has the authType structure and no secretKey. Adding such a test would guard against regressions to the idempotency guarantee, which is the key safety property for users upgrading through different version paths (e.g., 1.12.5 → 1.12.6 → 1.13).

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqubecloud
Copy link
Copy Markdown

@yan-3005 yan-3005 enabled auto-merge (squash) April 16, 2026 15:56
@github-actions
Copy link
Copy Markdown
Contributor

🟡 Playwright Results — all passed (21 flaky)

✅ 3638 passed · ❌ 0 failed · 🟡 21 flaky · ⏭️ 89 skipped

Shard Passed Failed Flaky Skipped
🟡 Shard 1 477 0 3 4
🟡 Shard 2 645 0 2 7
🟡 Shard 3 646 0 6 1
🟡 Shard 4 621 0 5 27
✅ Shard 5 614 0 0 42
🟡 Shard 6 635 0 5 8
🟡 21 flaky test(s) (passed on retry)
  • Features/DataAssetRulesDisabled.spec.ts › Verify the ApiEndpoint entity item action after rules disabled (shard 1, 1 retry)
  • Features/DataAssetRulesDisabled.spec.ts › Verify the Database entity item action after rules disabled (shard 1, 1 retry)
  • Pages/UserCreationWithPersona.spec.ts › Create user with persona and verify on profile (shard 1, 1 retry)
  • Features/BulkEditEntity.spec.ts › Glossary (shard 2, 1 retry)
  • Features/ChangeSummaryBadge.spec.ts › Automated badge should appear on entity description with Automated source (shard 2, 1 retry)
  • Features/RestoreEntityInheritedFields.spec.ts › Validate restore with Inherited domain and data products assigned (shard 3, 2 retries)
  • Features/RestoreEntityInheritedFields.spec.ts › Validate restore with Inherited domain and data products assigned (shard 3, 1 retry)
  • Features/RestoreEntityInheritedFields.spec.ts › Validate restore with Inherited domain and data products assigned (shard 3, 1 retry)
  • Features/RestoreEntityInheritedFields.spec.ts › Validate restore with Inherited domain and data products assigned (shard 3, 1 retry)
  • Features/RestoreEntityInheritedFields.spec.ts › Validate restore with Inherited domain and data products assigned (shard 3, 1 retry)
  • Features/RTL.spec.ts › Verify Following widget functionality (shard 3, 1 retry)
  • Pages/Customproperties-part2.spec.ts › entityReferenceList shows item count, scrollable list, no expand toggle (shard 4, 1 retry)
  • Pages/DataContracts.spec.ts › Create Data Contract and validate for Table (shard 4, 2 retries)
  • Pages/DataContracts.spec.ts › Create Data Contract and validate for ApiEndpoint (shard 4, 1 retry)
  • Pages/Domains.spec.ts › Rename domain with assets (tables, topics, dashboards) preserves associations (shard 4, 1 retry)
  • Pages/Domains.spec.ts › Subdomain rename does not affect parent domain and updates nested children (shard 4, 1 retry)
  • Pages/HyperlinkCustomProperty.spec.ts › should accept valid http and https URLs (shard 6, 1 retry)
  • Pages/Lineage/DataAssetLineage.spec.ts › verify create lineage for entity - Table (shard 6, 1 retry)
  • Pages/Lineage/LineageFilters.spec.ts › Verify lineage schema filter selection (shard 6, 1 retry)
  • Pages/Lineage/LineageRightPanel.spec.ts › Verify custom properties tab IS visible for supported type: searchIndex (shard 6, 1 retry)
  • Pages/Users.spec.ts › Permissions for table details page for Data Consumer (shard 6, 1 retry)

📦 Download artifacts

How to debug locally
# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

@yan-3005 yan-3005 merged commit cc06887 into main Apr 16, 2026
68 of 71 checks passed
@yan-3005 yan-3005 deleted the ram/webhook-migrations branch April 16, 2026 17:02
yan-3005 added a commit that referenced this pull request Apr 16, 2026
…#27438)

* feat(migration): add webhook secretKey to authType migration in v1130

* test(migration): add idempotency test for already-migrated webhook rows in v1130

(cherry picked from commit cc06887)
siddhant1 pushed a commit that referenced this pull request Apr 17, 2026
…#27438)

* feat(migration): add webhook secretKey to authType migration in v1130

* test(migration): add idempotency test for already-migrated webhook rows in v1130
siddhant1 added a commit that referenced this pull request Apr 28, 2026
* feat(ui): refactor AddDomainForm to use core components library

Migrate AddDomainForm from Ant Design to @openmetadata/ui-core-components
using HookForm, getField, and FieldTypes. Add onFocus prop support to
AutocompleteBase, add rules prop to FormField, and export form-field
components from the core library.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* feat(ui): enhance AddDomainForm Select, avatars, and glossary terms

- Switch FieldTypes.SELECT to use Select with Select.Item children
  for proper icon/avatar/supportingText support
- Pass fontSize through SelectContext so items respect the fontSize prop
- Show colored-initial avatars for users and team icons via Avatar
- Show tag color dots using Dot component on tag options
- Replace custom glossary terms autocomplete with MUIGlossaryTagSuggestion

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

* fix(ui): resolve ESLint, TypeScript, and review issues in AddDomainForm PR

Fix jsx-sort-props errors in form-field.tsx, replace `any` with `Domain`
type, use MutableRefObject to fix readonly ref assignment, add missing
fontSize to SelectContext providers, and restrict experts field to
user-only options matching original behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(ui): restructure form-field into typed, single-responsibility modules

Split the monolithic form-field.tsx (~1290 lines) into focused files:
- fields/color-picker-field.tsx and fields/icon-picker-field.tsx for standalone field components
- render-field-element.tsx for the central field type dispatch
- form-field.tsx reduced to thin Field/FormFields wrapper (~95 lines)

Replaced Record<string, unknown> FieldPropsMap with a properly typed interface,
eliminated all `as` casts and `unknown` types, and removed the unnecessary
select normalization layer (consumers pass correctly shaped SelectItemType data).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ui): store full SelectItemType objects in form state and fix TS errors

The form field system now stores entire SelectItemType objects instead of
just IDs, matching the data format expected by form submit handlers.
Also adds filterOption, onFocus, size, fontSize to FieldPropsMap and
fixes type compatibility in AddDomainForm.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ui): add missing MUIGlossaryTagSuggestion mock in AddDomainForm tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ui): resolve icon rendering and i18n issues in domain form

- Fix getEntityAvatarProps to resolve icon names via ICON_MAP as
  placeholderIcon, so icons like "Bank" render on domain/DP pages
  instead of showing the default placeholder
- Extract transformFormData from handleFormSubmit and reuse in
  validateFields so it returns API-ready payloads
- Remove hardcoded English strings in core-components form fields;
  accept labels as props for i18n support
- Fix unstable onBlur ref in icon-picker causing effect listener churn
- Remove no-op useMemo in icon-picker

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* lint

* fix(ui): CI failures on add-domain-form refactor

- lint/prettier autofix on core-components files flagged by CI
- wrap Domain Type field with data-testid="domainType" so Playwright
  can locate the new core-components Select trigger
- update fillDomainForm helper to click the Select trigger button and
  use exact option match instead of the old Ant combobox locator
- update DomainUIInteractions special-character validation test to
  assert the error text instead of the gone .ant-form-item-explain-error

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(playwright): resolve all domain form test failures after refactor

Fix 4 root causes: add missing domain-select testid, scope domainType
selector to form to avoid strict mode violation with KnowledgePanel,
use role-based selectors for tag options in new Autocomplete, and fix
special characters test to use :: which the regex actually rejects.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(playwright): update data product domain selector

* fix

* lint

* chore(ui): apply prettier fix to render-field-element

* fix(playwright): use add-domain-form testid for drawer visibility checks

The AddDomainForm root element testid was renamed from add-domain to
add-domain-form during the refactor, but drawer visibility assertions
in DataMarketplace and DataProductAndSubdomains specs still referenced
the old id.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(ui): align domain form coverImage type and drawer scroll-to-error

- Type coverImage as CoverImageFileValue | null to match the runtime
  value written by FieldTypes.COVER_IMAGE_UPLOAD and consumed by
  createEntityWithCoverImage.
- Switch useFormDrawerWithHook's scroll-to-error selector from the
  Ant Design class to [aria-invalid="true"] so validation failures
  land on the first invalid react-hook-form field.
- Move the Description wrapper's aria-invalid into the FormField render
  callback so the rich-text block participates in the same scroll path.

* fix(ui): restore domain cover image validation and stabilize tag test helper

- Reintroduce the size (5 MB) and dimension (800x400) checks that
  existed on the legacy MUI cover image upload by adding a react-hook-
  form validate rule on the coverImage field.
- Use tagFqn with an exact match in selectTagInTagSuggestion so search
  terms containing regex metacharacters cannot match unintended options.

* refactor(ui): extract submitAndClose helper for drawer submit flow

Consolidate the duplicated "await submit, close drawer, refresh list"
pattern into one helper in FormDrawerUtils so the six useFormDrawerWithHook
consumers no longer reference closeDrawer/refresh* from a useCallback that
runs before the drawer hook binds them. Error flows are preserved: if the
submit handler throws, closeDrawer/onSuccess are skipped and the drawer's
existing try/catch keeps the drawer open.

* fix(ui-core): forward onFocus through form-field renderer, drop unsafe cast

- Type FieldPropsMap.onFocus as FocusEventHandler so consumers can pass
  event-carrying or plain handlers without coercion.
- Destructure onFocus (and the 'sm'|'md' style size) out of the spread
  in renderFieldElement so they land only on Autocomplete, not on
  NativeSelect whose native size attribute expects a number.
- Thread onFocus through to Autocomplete so focus-triggered prefetch
  in AddDomainForm (tags, domain, owners, experts) actually fires.
- Explicitly type AutocompleteProps.onFocus and drop the cast in
  AutocompleteBase that the type mismatch was papering over.

* Update openmetadata-ui/src/main/resources/ui/src/components/Domain/AddSubDomainModal/AddSubDomainModal.component.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: use hookForm.handleSubmit() in useFormDrawerWithHook to preserve RHF submit lifecycle

Agent-Logs-Url: https://github.com/open-metadata/OpenMetadata/sessions/6f7c35d8-9f8f-4da6-8d0e-e8b78e33a732

Co-authored-by: siddhant1 <30566406+siddhant1@users.noreply.github.com>

* fallback

* fix(test): use stable testId to select tag option in TagSuggestion

The new Autocomplete.Item in @openmetadata/ui-core-components computes
the option's accessible name as `label + ' ' + supportingText`, so
`getByRole('option', { name: tagFqn, exact: true })` never matches when
a tag has a displayName. Revert to `getByTestId('tag-option-<fqn>')`,
which is still emitted by TagSuggestion on every Autocomplete.Item and
is unaffected by the supporting-text concatenation.

Fixes three TagSuggestion tests that timed out on all retries in
playwright-ci-postgresql (4, 6):
  - DataProducts.spec.ts:449 Create data product with tags using TagSuggestion
  - Domains.spec.ts:1196 Create domain with tags using TagSuggestion
  - Domains.spec.ts:1243 Create subdomain with tags using TagSuggestion

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update openmetadata-ui/src/main/resources/ui/src/components/Domain/AddDomainForm/AddDomainForm.component.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(domain-form): emit data-testid on tag option in AddDomainForm

The refactored AddDomainForm renders the tag input via the generic
Autocomplete renderer (getDefaultAutocompleteItems), which never passes
data-testid to Autocomplete.Item. As a result, the playwright helper
selectTagInTagSuggestion() could not find tag-option-<fqn> elements in
the dropdown, causing three TagSuggestion specs to time out:
  - DataProducts.spec.ts:449 Create data product with tags using TagSuggestion
  - Domains.spec.ts:1196 Create domain with tags using TagSuggestion
  - Domains.spec.ts:1243 Create subdomain with tags using TagSuggestion

Provide a custom renderItem on the tags field that mirrors the default
item rendering and forwards data-testid="tag-option-${item.id}" so the
existing test selector works against the new component path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* tests: fix flaky SystemCertificationTags test where Gold tag missing from dropdown after classification re-enable (#27415)

* tests: fix flaky SystemCertificationTags test where Gold tag missing from dropdown after classification re-enable

* nit

* Fix restrore entity inherited fields specs (#27414)

* increase data contracts test timeout

* fix failure in restroreEntityInheritedFields

* fix restrore entity inheritance specs

* remove unnecessary multiple domains check

---------

Co-authored-by: Shrabanti Paul <shrabantipaul@Shrabantis-MacBook-Pro.local>

* Fix flaky customizedetail page spec (#27420)

* increase data contracts test timeout

* fix flaky customizeDetailPage specs

---------

Co-authored-by: Shrabanti Paul <shrabantipaul@Shrabantis-MacBook-Pro.local>

* fix(test): unflake GlossaryPermissions team-based test (#27422)

* fix(test): assign role to team in P-11 team-based permissions test

The "Team-based permissions work correctly" test created a team and added
testUser to it, but never attached the new role to the team — so the user
inherited no permissions from team membership. The test relied on default
Organization conditional rules, which the frontend treats as no-permission
(only Access.Allow becomes truthy in PermissionsUtils). The glossary page
then rendered the no-permission placeholder and never fired
/api/v1/glossaries?fields=*, hanging visitGlossaryPage's waitForResponse
for the full 30s toPass budget.

Patch the team's defaultRoles after initializePermissions so the user
actually inherits Allow via team membership, matching the test's intent.

Local repro (full file × 10 repeats × 4 workers, retries=0):
- Before: 10/10 P-11 failures
- After:  10/10 P-11 pass, 93/93 overall pass

* fix(test): isolate P-11 to a dedicated user/team

Avoid mutating the file-shared testUser. Previously the fix added testUser
to a temporary team and attached an Allow role; if cleanup failed, the
elevated permissions would leak into subsequent tests sharing the same
worker.

Now P-11 creates its own UserClass + page, runs the team-permission
verification in isolation, and tears down the user, team, role, and policy
at the end. Other tests in the file see no state change from this test
beyond what initializePermissions/cleanupPermissions already do.

Local verification (full file × 10 repeats × 4 workers, retries=0):
93/93 pass, P-11 10/10.

---------

Co-authored-by: Siddhant <siddhant@MacBook-Pro-621.local>

* Fixes #24636: use test_metadata.kwargs['model'] to identify primary table for dbt test entity links (#27366)

* fix: use test_metadata.kwargs['model'] to identify primary table for entity links (issue #24636)

For dbt relationship tests with multiple upstream dependencies, the order of
tables in depends_on.nodes varies by database engine (Snowflake vs Unity Catalog).
The primary table being tested is explicitly specified in test_metadata.kwargs['model']
for generic tests, making this a reliable order-independent way to identify the correct
table for entity link generation. This fixes validation failures when columns exist in
the primary table but not in the referenced table.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Run formatter

* Apply Gitar-bot comments

---------

Co-authored-by: Claude <noreply@anthropic.com>

* Chore(UI): consolidated UI checkstyle fix commands and modify workflow comment (#27402)

* feat: add consolidated UI checkstyle commands for all and changed files

* update prt to pr

* test commit to fail ui-checkstyle

* update the comment

* Revert "test commit to fail ui-checkstyle"

This reverts commit ed056f0.

* Revert "update prt to pr"

This reverts commit 0666fa5.

* Worked on comments

* pull request target remove

* Revert "pull request target remove"

This reverts commit b61e98c.

* Worked on comments

* fix: add missing SecurityConfig init in reIndex and reIndexDI CLI commands (#27424) (#27425)

* fix(migration): revert webhook authType back to secretKey in v1126 and remove broken v1125 migration (#27427)

* fix(migration): add v1126 reverse migration to revert webhook authType back to secretKey

* fix(migration): remove migrateWebhookSecretKeyToAuthType from v1125 migration

* fix(test): remove migrateWebhookSecretKeyToAuthType references from v1125 migration tests

* fix(migration): address copilot review comments on v1126 migration

* fix(migration): case-insensitive bearer check and verify JSON content in v1126 tests

* fix(migration): remove unused constants from v1125 and add postgres path + SQL verification to v1126 tests

* feat(explore): redesign search export scope to export full tab results with accurate counts (#27354)

* feat(explore): redesign search export scope to export full tab results with accurate counts

* fix UI checkstyle and explore support

* fix checkstyle

* fix failing spec & code refactor

* add backend support for handling totalVotes export in explore page

* fix specs

* add skeleton loader

* Fix omjob pod/label naming length constraints (#27143)

* Fix omjob pod/label naming length constraints

- Fix SHA-256 hash byte formatting with & 0xff mask for proper
  2-hex-digit encoding per byte
- Enforce Kubernetes 63-character label limit via hash-based truncation
- Extract shared hash utility to HashUtils
- Add comprehensive tests for truncation, uniqueness, and edge cases

Fixes #27004

* Address review: fix hash bounds, add edge case tests, remove redundant substring

- Guard ensureValidLabelValue fallback against StringIndexOutOfBounds
- Add tests for separator-only inputs exercising the fallback path
- Remove redundant .substring() since HashUtils.hash() already returns 6 chars
- Use 253 (K8s DNS subdomain limit) instead of 260 in PodManagerTest
- Fix wrong assertion in LabelBuilderTest (podSelector has 2 entries)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: complete all code review issues for PR merge

- Add HTTP 409 idempotency handling in TableClass.create() for sharded Playwright tests
- Apply Java Spotless formatting to fix checkstyle violations
- Apply Playwright formatting: organize-imports, eslint --fix, prettier --write
- Resolve all 10 code review findings:
  ✅ StringIndexOutOfBoundsException in hash truncation (already fixed)
  ✅ Redundant substring operation (already fixed)
  ✅ Duplicate hash code extraction (already done)
  ✅ Playwright 409 conflict handling (now added)
  ✅ Java formatting compliance (now applied)
  ✅ TypeScript formatting compliance (now applied)

PR is now ready for merge with all CI checks expected to pass.

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test:Added missing test for ontology (#27423)

* test:Added missing test for ontology

* Added other missing part of test

* fix lint issue

* fix(workflows): make Flowable schema upgrades idempotent to survive partial migrations (#27234)

* fix(workflows): make Flowable schema upgrades idempotent to survive partial migrations

Fixes #26048.

When the server crashed mid-startup during a Flowable schema upgrade, the DB
was left in a partially-migrated state. On restart, Flowable re-ran the same
DDL and failed on already-existing objects (indexes, tables, columns), permanently
wedging both the server and migrate --force.

Changes:
1. WorkflowHandler: webserver now uses DB_SCHEMA_UPDATE_FALSE — it validates the
   schema but never runs DDL. Only migrate CLI uses DB_SCHEMA_UPDATE_TRUE.
2. OpenMetadataOperations: explicit WorkflowHandler.initialize(config, true) inside
   the migrate command so Flowable DDL always runs during migration.
3. WorkflowHandler: catches FlowableWrongDbException on webserver startup and
   rethrows with an actionable message directing the operator to run migrate.
4. IdempotentDdlDataSource + IdempotentDdlStatement: JDBC DataSource wrapper used
   exclusively in migration context. Intercepts execute(sql) for CREATE INDEX,
   CREATE TABLE, and ALTER TABLE ADD COLUMN and pre-checks existence via standard
   DatabaseMetaData (getIndexInfo, getTables, getColumns) before executing. If the
   object already exists it logs a skip and returns — no SQL state codes, no string
   matching, works on MySQL and PostgreSQL.

Unit tests cover schema-update mode selection in both contexts.

* fix(workflows): address review comments on idempotent DDL wrapper

- Extract shouldSkip() helper; apply idempotency checks to all execute()
  and executeUpdate() overloads, not just execute(String)
- Tighten ALTER TABLE regex with negative lookahead to exclude SQL keywords
  (CONSTRAINT, PRIMARY, UNIQUE, FOREIGN, CHECK, INDEX, KEY) from being
  matched as column names
- IdempotentDdlDataSource now wraps a DataSource delegate instead of calling
  DriverManager directly; uses migrationDataSource() helper in WorkflowHandler
  to resolve from existing DataSource or JDBC params
- Fix InvocationTargetException wrapping in Connection proxy — unwrap cause
  so callers receive the original SQLException
- Wrap all createStatement() variants in the proxy, not just the no-arg form
- Contextual error message in WorkflowHandler — distinguish between server
  startup and migration context
- Add IdempotentDdlStatementTest: 11 tests covering skip/execute for
  CREATE INDEX, CREATE UNIQUE INDEX, CREATE TABLE, ALTER TABLE ADD COLUMN,
  keyword-guarded ALTER TABLE, executeUpdate overload, and pass-through

* fix(workflows): include DB/library versions in FlowableWrongDbException message

* test(workflows): add IdempotentDdlDataSourceTest for proxy wrapping and exception surfacing

* test(workflows): assert exception identity in proxy exception-surfacing tests

* fix(workflows): catalog-aware identifier normalization in IdempotentDdlStatement

On MySQL with lower_case_table_names=0 (default on Linux), table names are
stored as-is and catalog=null metadata lookups can miss existing objects.

- Use connection.getCatalog() for all getIndexInfo/getTables/getColumns calls
- Normalize identifiers via DatabaseMetaData.storesLowerCaseIdentifiers() /
  storesUpperCaseIdentifiers() instead of unconditional toLowerCase()
- stripIdentifierQuotes() handles backtick, double-quote and bracket quoting
- extractObjectName() handles schema-qualified names (schema.table)
- columnExists now iterates and normalizes COLUMN_NAME from ResultSet
- Test: added MySQL uppercase storage case to IdempotentDdlStatementTest

* fix(workflows): null guard in shouldSkip, drop-create Flowable init, robust test indexing

- shouldSkip() returns false immediately for null SQL, preserving JDBC contract
  (delegate handles null and throws the driver's own error)
- drop-create command now calls WorkflowHandler.initialize(config, true) after
  native migrations so it produces a fully startable DB including Flowable tables
- WorkflowHandlerSchemaUpdateTest: replace brittle get(1) with getLast() so the
  test is not sensitive to how many StandaloneProcessEngineConfiguration instances
  are constructed before initializeNewProcessEngine runs

* Move ontology/glossary relation migration from 1.14.0 back to 1.13.0 (#27431)

* Move ontology/glossary relation migration from 1.14.0 back to 1.13.0

Ontology feature will ship in 1.13.0, not 1.14.0. Move the glossary term
relation migrations (relationType backfill, settings insert, stale
relatedTerms strip, conceptMappings backfill) back to the 1.13.0
postDataMigrationSQLScript for both MySQL and PostgreSQL.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Restore empty 1.14.0 SQL migration files for Java migration framework

The V114 MigrationUtil.java package requires the 1.14.0 migration
directory to exist with SQL files for the migration to be picked up.
Keep them as empty files (matching convention of other versions with
no post-data SQL).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Add schemaChanges.sql and comment all 1.14.0 SQL migration files

Add both schemaChanges.sql and postDataMigrationSQLScript.sql for
mysql and postgres with a comment explaining the directory is required
for the V114 Java migrations to be picked up by the migration framework.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Fix missing trailing newline in postgres postDataMigrationSQLScript

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* address feedback

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>

* feat(migration): add webhook secretKey to authType migration in v1130 (#27438)

* feat(migration): add webhook secretKey to authType migration in v1130

* test(migration): add idempotency test for already-migrated webhook rows in v1130

* Add backticks to table name in mysql median function (#27406)

* Fixes #26198: Filter constraints referencing non-existent columns for Redshift AUTO-distribution MVs (#27016)

Co-authored-by: Teddy <teddy.crepineau@gmail.com>

* Chore(deps): Bump org.eclipse.jetty:jetty-http in /openmetadata-service (#27372)

Bumps org.eclipse.jetty:jetty-http from 12.1.6 to 12.1.7.

---
updated-dependencies:
- dependency-name: org.eclipse.jetty:jetty-http
  dependency-version: 12.1.7
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>

* Fixes #21953, #23338, #27380: upgrade collate-sqllineage to >=2.1.1 with regression tests (#27413)

* deps(ingestion): upgrade collate-sqllineage to >=2.1.1 with expanded lineage test coverage

* address copilot comments

* style(ui): apply prettier fix in combobox.tsx

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ui): address Copilot review comments on domain form refactor

- Memoize fetchDataProducts with useCallback and add it to
  onDataProductCreateSuccess deps (stale closure fix)
- Correct misleading JSDoc in FormDrawerUtils.submitAndClose
- Pass translated labels to IconPickerField in AddDomainForm
- Narrow Alert helperText guard to typeof string to prevent empty title

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(ui-core): guard Alert against empty-string helperText

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style(ui-core): prettier fix on form-field.tsx

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update openmetadata-ui/src/main/resources/ui/src/components/Domain/AddDomainForm/AddDomainForm.component.tsx

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix(ui-core): forward onBlur in Autocomplete fields and fix ReactNode helperText alert

- Pass onBlur handler to Autocomplete in renderFieldElement so RHF marks
  fields as touched and blur-based validation fires for all AUTOCOMPLETE_FIELD_TYPES
- Fix ALERT helperText rendering in Field to handle ReactNode: string values
  go to Alert title, non-string nodes render as Alert children

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(ui-core): remove MULTIPLE_SELECTION_FIELD_TYPES set, require explicit multiple prop

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style(ui-core): fix prettier lint on render-field-element

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(ui): add unit tests for transformDomainFormData helper

Covers DOMAIN / SUBDOMAIN / DATA_PRODUCT payload shapes, tag and
glossary merging, style packaging, owner / expert extraction, and the
parent-domain FQN fallback.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(ui-core): use Button, Typography, Box in form-field primitives

Addresses @anuj-kumary's review on #26951: replace native <button>,
<span>, and flex-column <div> with core-library components wherever
they fit.

- color-picker-field: swatch <button> → Button; empty-state <span> →
  Typography; outer flex <div> → Box.
- icon-picker-field: tile <button> + trigger <button> → Button; all
  user-visible <span> (fallback "?", first-letter initial, empty state,
  URL panel label) → Typography; URL-panel wrapper → Box.
- form-field: field wrapper <div> → Box.
- render-field-element: COVER_IMAGE_UPLOAD wrapper <div> → Box.
- AddDomainForm (consumer): four flex-container <div> (icon+color row,
  name+displayName row, description wrapper, CTA row) → Box.

Grid layouts, positional wrappers, and inline-flex label spans are left
as native elements — Box/Typography would change layout or DOM
semantics.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style(ui-core): apply checkstyle autofix and widen icon picker dropdown

- Runs eslint/prettier autofix that CI's lint-core-components expected.
- Bumps the icon-picker dropdown to 22rem and grid gap to gap-4 so the
  6-column icon grid has visible breathing room.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(ui): mock Autocomplete, Avatar, Box, Dot in AddDomainForm.test

Refactored AddDomainForm renders Box unconditionally and Autocomplete / Avatar / Dot in the tag-suggestion and user-team paths. The existing jest.mock of @openmetadata/ui-core-components omitted these, so every render through AddDomainFormHarness threw "Element type is invalid", failing all 11 describe('AddDomainForm') tests on SonarCloud.

Add passthrough stubs (Autocomplete with .Item, Avatar, Box, Dot) to the mock so the component tree resolves.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(glossary): drop flaky mutex batch from "Add and Remove Assets"

The first batch in this test added two siblings from a mutuallyExclusive
glossary and relied on the server rejecting the save so the UI stayed
empty and `add-tag` remained clickable for the next batch.

TreeAsyncSelectList.handleChange now drops mutex siblings client-side
once its glossaries-list fetch resolves, so only one term reaches the
server, it saves successfully, tags become non-empty, and `add-tag`
is replaced by `edit-button`. The test then times out waiting for
`add-tag`. Whether the filter kicks in depends on a race between the
dropdown's `/api/v1/glossaries?fields=mutuallyExclusive` call and the
user clicks, which is why the test sometimes passed and sometimes
timed out at 3 minutes under CI load.

Remove the mutex batch (and the now-unused glossary1/term1/term2
fixtures). The mutex flow was never actually asserted here — it was
only used as a side channel to keep tags empty. The remaining
non-mutex batch + chart-level tagging + asset verification still cover
the add/remove-assets behavior end-to-end.

Local: 5/5 passes in ~7s each (was timing out at 180s).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(ui-core): drop looksLikeImageSource helper from IconPickerField

URL-detection for the custom-icon preview no longer lives inside the core
picker. hasCustomImage (allowUrl + non-empty value + no selectedItem) is
enough to decide whether to render the <img>; matches main's identity of
keeping image-validity rules in consumer utils, not in the picker itself.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor: trim PR scope back to MUI → Untitled UI swap only

Strip changes that weren't required by the form-library migration so the
diff against main is purely the MUI → Untitled UI swap:

- utils/IconUtils.tsx + test: restore getEntityAvatarProps to main. The
  ICON_MAP[iconURL] lookup for stored icon names is new rendering
  behaviour, not needed by the form refactor.
- AddDomainForm.component.tsx: drop the getImageDimensions helper and
  the coverImage rules.validate block. On main that size/dimension
  check lived inside MUICoverImageUpload via maxSizeMB/maxDimensions
  props — keep the same principle (backend rejects on upload), don't
  reimplement it in the form.
- playwright DomainUIInteractions.spec.ts: restore the 'Invalid@Name#Test'
  input and .ant-form-item-explain-error selector. The branch had
  swapped both the test input and the assertion target; neither was
  required by the drawer testid migration.
- playwright Glossary.spec.ts: restore to match main. The 67-line
  flaky-batch deletion is unrelated to the Add Domain form migration.
- core form-item-label.tsx / form-field.types.ts / form-field.tsx:
  drop the unused isBeta / betaLabel props — no consumer sets them.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ui): don't POST empty-string style fields from Add Domain form

Hook-form defaults color / iconURL to '', which made the submit payload
include `style: { color: '', iconURL: '' }`. Because getEntityAvatarProps
uses ?? (not ||) for the brand-600 fallback, empty strings defeated the
default and avatars rendered with no background. Strip empty values at
the submit boundary so the POST payload matches what the Ant-based form
on main produced.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ui): render picked icon names in entity avatars

The Untitled UI icon picker stores preset icon selections as the icon's
name (e.g. 'Bank') in entity.style.iconURL. Main's getEntityAvatarProps
only handled http/absolute-path URLs, so any stored icon name fell through
to the entity-type default — the user's picked icon never showed up in
list views, tree views, cards, or detail headers.

Look up non-URL iconURL values in ICON_MAP and render the matching icon
component as the avatar's placeholderIcon. Unknown names still fall back
to the entity-type default. Restore the two tests that cover both paths.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(domain): fix special-chars name validation assertion

Input didn't actually violate ENTITY_NAME_REGEX (`/^((?!::).)*$/`), and
the selector targeted the old AntD error class which the HookForm
refactor removed. Use a `::` input and match the message via text.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(ui-core): extract entity color palette to shared constant

Dedupes the 14-color swatch palette between ColorPickerField and
its icon-picker sibling into a single ENTITY_PALETTE_HEX export
under @/colors.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(ui-core): drop DEFAULT_COLOR_OPTIONS alias and inline icon styles

Use ENTITY_PALETTE_HEX directly instead of re-exporting it as
DEFAULT_COLOR_OPTIONS, and express icon display/stroke-width via Tailwind
utilities so the picker fields no longer carry an inline style object.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style(ui-core): apply prettier to icon-picker-field

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(ui-core): drop redundant case-folding in ColorPickerField

normalizeHexColor canonicalizes hex strings to uppercase, so both sides
of every comparison were already in the same case. Replace the
.toLowerCase() pairs with plain equality (and palette.some with
palette.includes), and drop the redundant array copy.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Siddhant <siddhant@MacBook-Pro-407.local>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Siddhant <siddhant@MacBook-Pro-457.local>
Co-authored-by: Siddhant <siddhant@MacBook-Pro-529.local>
Co-authored-by: Siddhant <siddhant@MacBook-Pro-621.local>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Anujkumar Yadav <anujf0510@gmail.com>
Co-authored-by: shrabantipaul-collate <shrabanti.paul@getcollate.io>
Co-authored-by: Shrabanti Paul <shrabantipaul@Shrabantis-MacBook-Pro.local>
Co-authored-by: Eugenio <eugenio.donaque@getcollate.io>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Adrià Manero <adria.estivill@getcollate.io>
Co-authored-by: Ram Narayan Balaji <81347100+yan-3005@users.noreply.github.com>
Co-authored-by: Harsh Vador <58542468+harsh-vador@users.noreply.github.com>
Co-authored-by: Darshan Rajput <darshan.rajput123091@marwadiuniversity.ac.in>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Rajdeep Singh <157990661+RajdeepKushwaha5@users.noreply.github.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mohit Tilala <63147650+mohittilala@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants